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1. INTRODUCTION 

The agro-climatic and atmospheric conditions of our country are perfect for agriculture. Our country’s 
economy is also a preponderantly agricultural-based economy. Most of all kinds and varieties of agriculture 
can be seen here. Agriculture promotes 65% of the workforce and earns 20% of our exports. India is the 
extensive producer of fruits in the world (45 million tonnes) with a global share of over 11% and the second- 
largest producer of vegetables (81 million tonnes) with a worldwide share of over 16%. The automation of the 
fruit/vegetable sorting and packaging industry is one of the important milestones in this market. Farmers do 
traditional scrutiny of quality and handpicking to sort and grade fruits/vegetables. But these have many 
liabilities like sluggishness, time-consuming, and irreconcilable [1]-[3]. The increasing need for processed 
fruits/vegetables often acts by decreasing the processing and packaging time of these fruits/vegetables. By 
decreasing the processing cycle and utilizing automated machinery for fruit/vegetable sorting and packaging it 
can often be achieved [4]-[6]. So, the use of automated machinery has finally decreased the processing time 
and increased the quality of fruits/vegetables and products manufactured with this [3]-[5]. In spite of the fact 
that automatic machines are available to ease the issue, the problem is that they have a very extensive set up 
which raises the cost [7]-[10]. Hence my vision is to develop a system that is economical and efficient so that 
every single fruit/vegetable farmer can set up his own fruit/vegetable sorting and packaging facility and will 
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avail himself of his own market, bringing self-reliability to farmers. Initially, the system is focused on the 
sorting and packaging [11 ]-[15] of apples on the basis of physical appearance and sizing (weight and diameter). 
But can also be used for other spherical fruits/vegetables such as pears, oranges, and tomatoes. 


2. STRUCTURE DESCRIPTION 

The whole design of the machine is based on a simple ideology “of making things cheaper and 
simple”. The structure of the machine is divided into three sections which are dependent on the process, but 
structure wise it is independent of each other that makes the whole machine easy to relocate from one place to 
another. The frame of the machine is made up of Marandi wood (Melia azedarach) or white cedar timber due 
to ease on machining, also it is resistant to decay and termite properties, which not only have given a rigid 
structure but also have shown steadiness in drilling holes. While operation it didn’t require high tolerance and 
can be worked on low tolerance to achieve goals. 

The alignment section is the initial part of the assembly where fruits/vegetables are poured onto the 
conveyor belt and all apples start moving. But to go to another section it requires that fruits/vegetables should 
be aligned properly. Hence, two flanges of length 49.28 cm and angle 21.24° from the horizontal axis (when 
seen from top view) as shown in Figure 1, makes a nozzle like structure with converging section (inlet=36 cm 
and outlet=15 cm) [16], [17]. 





Figure 1. Top view of machine 


Rollers are made up of polypropylene rods which are very cheap and light in weight. Each roller has 
a 15 mm hole is drilled throughout the propylene rod. At the periphery of each roller, a boring is done on both 
sides, which is equal to the outer diameter of the bearing (6202) used. A rod is passed through rollers and pillar 
on which the rollers are mounted to fix the rod, this makes the driver roller of the conveyor. The driving roller 
is designed with a twist, where shaft which is to be passed from the roller is bisected into two equal parts. First 
part is centrally bored so that the motor can be directly fixed and a conveyer belt is placed over both the rollers. 

The sorting section is one the main part of the assembly which acts as a decision-maker [18]-[22] for 
the next process. To transfer the apple after scanning, forward angular movement is required about an 
instantaneous center which sends the apple to the next station, for that motion, a CG rod is passed through the 
center, which performs two functions initially, it is balancing the structure and also connected with the motor 
which will attract it towards itself to perform clockwise instantaneous motion. The packaging section is similar 
to the alignment section, the only difference is that it has a path decider deflector of length 25.74 cm as shown 
in Figure 1 which is controlled upon the feedback of the sensing section. This deflector is connected with the 
servo motor and the servo motor moved as per the signal receives from Arduino UNO and accordingly, fruits 
are dropped in the packaging cartoons. 


3. PROCESS FLOW 

Figure 2 shows the proposed design of the customized sorting and packaging machine which can sort 
and pack various spherical fruits/vegetables such as pears, oranges, and tomatoes. Figure 3 shows the sequence 
diagram of aliging section and Figure 4 shows the sequence diagram of inspection and packaging section which 
should be controlled through a synchronized arrangement of Arduino UNO and Raspberry Pi along with servo 
motors, IR sensors and DC motors with a motor driver module i.e. L293D which is based on H-Bridge circuit. 
Firstly in the conveyor and aligning section fruits/vegetables are put on the conveyor belt, it will move along 
with the conveyor and passes through the two flanges which align each fruit/vegetable in a single line and then 
the fruits/vegetables goes towards inspecting platform of the inspection section, consisting of a camera at the 
top, two rollers which rotate each fruit/vegetable at 360°, and IR sensor which detects the fruit/vegetable on 
the inspecting platform and servo motor connected to a CG rod which tilt the fruit/vegetable towards the 
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conveyor of inspecting section and simultaneously pack them using a synchronous packaging arrangement 
consisting of a deflector which can be move using servo motor and pack each fruit in an ordered arrangement. 
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Figure 2. Proposed design of the machine 


Figure 2 also shows the inspecting platform of the inspection section, we can also call it as black-box 
because inside, whole box is totally black and illuminated with lights with a focused beam. In this section, 
there are two rollers that rotate the fruit/vegetable in 360° such that the camera can take images of the whole 
fruit/vegetable. There is an IR sensor in the middle of the two rollers which detects the presence of 
fruit/vegetable on the inspecting platform [18]-[22]. Now, as soon as the presence of fruit/vegetable is detected 
the camera starts taking five images of the fruit/vegetable in such a way that the five images will cover the 
maximum area of the fruit/vegetable, as the camera and rollers are both synchronized by the raspberry pi and 
Arduino UNO. Now the images are processed by the raspberry pi using an algorithm that is designed in Open 
CV library of python. Now, a servo motor is attached in the inspecting platform which tilts the inspecting 
platform towards high and low-quality packing conveyors and the fruit/vegetable will packed [23 ]-[25]. 
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Figure 3. Sequence diagram of aligining section 
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Figure 4. Sequencing diagram of inspection and packaging section 


4. PROPOSED IMAGE PROCESSING ALGORITHM 
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When an image is captured by the camera than every image will be simultaneously processed using 
the proposed algorithm and an output value is obtained. The proposed algorithm has been designed in python 
using open CV and numpy libraries [5], [6]. Here, the concept of changing color spaces and object tracking is 
used through which a range of colors other than the range of colors that we need to detect has been masked 
using Bitwise-AND operation. Figure 5 shows the flow chart of the proposed image processing algorithm in 


which 


— Using the function imread an image will be read which is stored in the working directory and store the 


image in a variable. 


— Now the image will be scaled and resized, so for this OpenCV comes with a function. Now for this, a 


scaling factor has to be specified. 


— Now the image will convert from BGR to HSV color-space because it is easier in HSV to represent any 
color than in BGR color-space [13]-[15]. RGB is an array matrix of color-space pixels, where each color 
pixel is a trio corresponding to the red, green, and blue components of an RGB image at a specific spacial 
location wheare as in HSV format it divides the color into 3 components hue, saturation, and value. H (hue) 
differentiate among the perceived colors, such as red, yellow, green, and blue, V (value) represents the 
brightness of a color and S (saturation) refers to how much the amount of white light mixed with a hue [17]. 
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Figure 5. Flow chart of the proposed image processing algorithm 
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— Now, with the use of Bitwise-AND operator and the upper and lower threshold value we will mask those 
values which we do not require [18]-[21]. And the discarded colors are the greenish, yellowish or some 
black spots, these kinds of colors are discarded or masked. 

— Now, after the Bitwise-AND operator is used the output image is a black and white image, which means 
the matrix of the image contains only zero and one as elements. The discarded portion is white and the rest 
are black. So, we can calculate how much white part is present in the image, then we can identify how much 
good an apple will be. So, by using a numpy array add the matrix of the output image, all the ones will be 
added [22 ]|-[25]. And we get a value i.e. quality score at the output. Now as higher as this score is, the more 


bad quality fruit/vegetable it will be. 


5S. EXPERIMENT AND DISCUSSION 


5.1. Materials and methods 


Through this machine, we can sort and pack any kind of spherical fruit but initially, apples are used 
in this experiment which we bought directly from the market. The quality of the external appearance of apples 
is classified into two categories: high, A, where more than 60% of the surface is deep red and B, low, where 
less than 60% of the surface is red. Fruit sorting procedure is given below: At first according to the grading 
criteria the samples are sorted manually into two groups (A and B), and 30 typical samples are taken from each 
group which makes one complete set of 60 apples in which 30 are from both category. The experiment will be 
done into three sets, so total 180 apples. At a time one set will be passed, which has been shown in Tables 1 
and 2. Now, secondly, start the fruit sorting system and the processing module begins to work. Set 1 will be 
put on the conveyor line, all 60 apples will be processed and sort. When one set is complete another set 1.e. Set 
2 will be put on the conveyor line and same as Set 3. The processing results will be shown in Table 1 and 2. 


Table 1. Manual classification of apples 
CATEGORY 


HIGH (A) 
LOW (B 


Table 2. Classification pf apples 


SAMPLE SET CATEGORY SAMPLE SET 
SET1 SET2 SET3 SET1 SET2 SET3 
30 30 30 HIGH (A) 28 29 31 
30 30 30 LOW (B 32 31 29 
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5.2. Classification result of apple 

The classification of apple results is shown in Table 2. The result shows that very small part of apple 
surface is ungoverned, so some part of the apple is invariant during processing due to lighting conditions. The 
result of the processing system is shown in the Figure 6. 





Figure 6. Processing system result 


5.3. Quality score analysis 

When all 180 apples where processed so for each apple a quality score should be calculated based on 
the processing algorithm implemented in the processing section. Figure 7 shows the quality score graph of all 
180 samples. It is clearly shown from graph that the points below 200000 are so close to each other, while the 
points above 200000 are least congested. Now on comparing the results with the manual sorting we can 
conclude that those apples who has a quality score of below 200000 will be high quality and those having 
quality score above 200000 will be of low quality. Hence, 200000 can be taken as quality threshold. 
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Figure 7. Quality score graph 
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6. CONCLUSION 

An automatic, synchronized and customized fruit/vegetable sorting and packaging system has 
introduced in this paper. It aligns each fruit in a single line; inspect them from 360° view, according to the pre- 
defined standards. Sort high-quality and low-quality products based on the quality score and concurrently pack 
them separately without any human intervention. 


REFERENCES 

[1] R. Dalai, and K. K. Senapati, “A Vision based Smart Multihead Weighing Machine for Automated Food Packager 
System,” 2019 IEEE R10 Humanitarian Technology Conference (R10-HTC)(47129), Nov. 2019, doi: 10.1109/R10- 
HTC47129.2019.9042430. 

[2] N. D. Thong, N. T. Thinh, and H. T. Cong, “Mango Sorting Mechanical System Combines Image Processing,” 2019 
IEEE 7th International Conference on Control, Mechatronics and Automation (ICCMA), Nov. 2019, doi: 
10.1109/ICCMA46720.2019.89887 14. 

[3] M. A. H. Ali, and K. W. Thai, “Automated Fruit Grading System,” 2017 IEEE 3rd International Symposium on 
Robotics and Manufacturing Automation (ROMA), Sept. 2017, doi: 10.1109/ROMA.2017.8231734. 

[4] Yogesh, P. Singhal, A. K. Dubey, and A. Goyal, “A Comparative Approach for Image Segmentation to Identify the 
Defected Portion of Apple,” 20/7 6th International Conference on Reliability, Infocom Technologies and 
Optimization (ICRITO), Sep. 2017, doi: 10.1109/ICRITO.2017.8342499. 

[5] A. Mordvintsev and K. Abid, “OpenCV-Python Tutorials Documentation,” Release I, 2017. 

[6] K. Y. Win and S. Choomchuay, “Automated segmentation of cell nucleiin cytology pleural fluid images using OTSU 
thresholding,” 2017 International Conference on Digital Arts, Media and Technology(ICDAMT), Mar. 2017, 
doi: 10.1109/ICDAMT.2017.7904925. 

[7] P. Ganesan, B. S. Sathish and G. Sajiv, “Automatic segmentation of fruitsin CIELuv color space image using hill 
climbing optimization and fuzzyC-Means clustering,” 2016 World Conference on Futuristic Trends inResearch and 
Innovation for Social Welfare (Startup Conclave), Mar. 2016, doi: 10.1109/STARTUP.2016.7583960. 

[8] Yogesh and A. K. Dubey, “Fruit defect detection based on speeded uprobust feature technique,” 20/6 Sth 
International Conference onReliability, Infocom Technologies and Optimization (Trends and FutureDirections) 
(ICRITO), 2016, pp. 590-594, doi: 10.1109/ICRITO.2016.7785023. 

[9] J. Pan et al., “Image segmentation based on 2D OTSU and simplifiedswarm optimization,” 2016 International 
Conference on MachineLearning and Cybernetics (ICMLC), Jul. 2016, doi: 10.1109/ICMLC.2016.7873020. 

[10] C. Kumar, S. Chauhan, R. N. Alla and H. M. Gurram, “Classifications of Citrus Fruit Using Image Processing -GLCM 
Parameters,” 2015 IEEE International Conference on Communications, Nov. 2015, doi: 10.1109/ICCSP.2015.7322820. 

[11] Seema, A. Kumar, and G. S. Gill, “Automatic Fruit Grading and Classification System Using Computer Vision: A 
Review. Proceedings,” 20/5 2nd IEEE International Conference on Advances in Computing and Communication 
Engineering, May 2015, doi: .10.1109/ICACCE.2015.15. 

[12] X. Song and L. Yang, “The Study of Adaptive Multi ThresholdSegmentation Method for Apple Fruit Based on the 
Fractal Characteristics,” 2015 8th International Symposium on Computational Intelligence and Design (ISCID), 
Dec. 2015. 

[13] C. S. Nandi, B. Tudu, and C. Koley, “A Machine Vision-Based Maturity Prediction System for Sorting of Harvested 
Mangoes,” IEEE Transactions On Instrumentation And Measurement, vol. 63, no. 7, Jul. 2014, doi: 
10.1109/TIM.2014.2299527. 

[14] H. Toylan and H. Kuscu, “A Real-Time Apple Grading System Using Multicolor Space,” Hindawi Publishing 
Corporation The Scientific World Journal, vol. 2014, Jan. 2014, doi: 10.1155/2014/292681. 

[15] Y. Gan and Q. Zhao, “An effective defect inspection method for LCD using active contour model,” IEEE Trans. 
Instrum. Meas., vol. 62, no. 9, pp. 2438-2445, Sep. 2013, doi: 10.1109/TIM.2013.2258242. 

[16] R. S. Jadhav, and S. S. Patil, “A Fruit Quality Management System Based On Image Processing,” 20/3 IOSR Journal 
of Electronics and Communication Engineering, vol. 8, no. 6, pp. 1—5, Jan. 2013, doi: 10.9790/2834-0860105. 

[17] Q. Li and S. Ren, “A real-time visual inspection system for discrete surface defects of rail heads,” IEEE Trans. 
Instrum. Meas., vol. 61, no. 8, pp. 2189-2199, Aug. 2012, doi: 10.1109/TIM.2012.2184959. 

[18] O. M. Ben Saeed, et al., “Classification of oil palm fresh fruit bunches based on their maturity using portable 
four-band sensor system,” J. Comput. Electron. Agricult, vol. 82, pp. 55-60, Mar. 2012, 
doi: 10.1016/j.compag.2011.12.010. 

[19] P. Deepa and S. N. Geethalakshmi, “Improved watershed segmentation for Apple fruit grading,” 2011 International 
Conference on Process Automation, Control and Computing, Jul. 2011, doi: 10.1109/PACC.2011.5979003. 

[20] A. Vallan and F. Molinari, “A vision-based technique for lay length measurement of metallic wire ropes, ” IEEE 
Trans. Instrum. Meas., vol. 58, no. 5, pp. 1756-1762, May 2009, doi: 10.1109/TIM.2009.2012953. 

[21] M. Larrain, A. R. Guesalaga, and E. Agosin, “A multipurpose portable instrument for determining ripeness in wine 
grapes using NIR spectroscopy,” IEEE Trans. Instrum. Meas., vol. 57, no. 2, pp. 294-302, Feb. 2008, 
doi: 10.1109/TIM.2007.910098. 

[22] J. Brezmes, et al., “Evaluation of an electronic nose to assess fruit ripeness,” JEEE Sensors Journal, vol. 5, no. 1, 
pp. 97-108, Feb. 2005, doi: 10.1109/JSEN.2004.837495. 

[23] G. Feng and C. Qixin, “Study on Color Image Processing Based Intelligent Fruit Sorting System,” Proceedings of 
the 5th World Congress on Intelligent Control and Automation, Jun. 2004, doi: 10.1109/WCICA.2004.1343622. 


TELKOMNIKA Telecommun Comput El Control, Vol. 19, No. 4, August 2021: 1326 - 1333 


TELKOMNIKA Telecommun Comput El Control O 1333 


[24] P. Moon, and G. De Jager, “An Heuristic Graph Searching Algorithm To Find The Boundary Of Apple Images,” 
IEEE Proceedings of Communications, Sept. 1992, doi: 10.1109/COMSIG.1992.274278. 

[25] M. A. H. Ali, M. Mailah, H. H. Tang, S. Kazi, “Visual inspection of cylindrical product’s lateral surface using 
cameras and image processing,” International Journal of Mathematical Models and Methods in Applied Sciences, 
vol. 6, no. 2, pp. 340—34, Jan. 2012. 


BIOGRAPHIES OF AUTHORS 


Ashish Bhatnagar is an undergraduate electronics and communication-engineering student 
pursuing B.Tech at ABES Institute of Technology, Ghaziabad. His area of interest 1s 
embedded systems, Internet of Things and system programming. He has done trainings in 
AVR & PIC microcontrollers, Raspberry Pi and Internet of Things 


Rachit Patel is B.Tech (ECE), M.Tech (ECE) from Jaypee university of information and 
technology, Waknaghat, Solan, H.P. He is pursuing Ph.D from JIIT, Noida. He has total 10+ 
years of experience in field of education and research. His area of interest is analog digital 
VLSI, DSP, image processing and Internet of Things. He has been published 15 research 
papers in reputed conference and journals. He has been guided so many students at graduation 
level. 


Manan Gupta is an undergraduate mechanical engineering student pursuing B.Tech at ABES 
Institute of Technology, Ghaziabad. His area of interest 1s industrial automation. He has done 
trainings in Autocad 3D, PIC & HMI configuration of seimens and Allan Bradley. 


Mayank Pal is an undergraduate mechanical engineering student pursuing B.Tech at ABES 
Institute of Technology, Ghaziabad. His area of interest is designing and programming. He has 
done trainings in AutoCAD, Solidworks, Pneumatics and Automation. 


Lakshay Kumar is an undergraduate mechanical engineering student pursuing B.Tech at 
ABES Institute of Technology, Ghaziabad. His area of interest 1s aeronautical and aerospace 
engineering. He has done trainings in AutoCAD, Pneumatics, automation and ANSYS 
software. 





Customized sorting and packaging machine (Ashish Bhatnagar) 


